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Foreword 


This  study  was  conducted  in  response  to  a  request  from  the  Chief  of  Naval  Personnel  (PERS- 
23)  to  validate  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  and  the  joint-service 
English  Diagnostic  Test  (EDT)  as  selection  criteria  for  the  Basic  Journalist  (JO)  Class  “A”  school. 
The  study  was  requested  in  review  of  a  proposal  to  administer  the  EDT  at  the  Military  Entrance 
Processing  Stations  (MEPS)  as  a  selection  screen  rather  than  at  the  JO  school  as  a  remediation  tool. 
This  study  was  completed  and  submitted  in  a  letter  report  in  November  1993  (Held,  1993). 

The  investigation  was  sponsored  by  PERS-23  and  funded  by  reimbursable  Work  Unit 
93WRPS578.  Results,  which  are  published  at  this  time  for  archival  purposes,  are  intended  for  use 
by  the  Bureau  of  Naval  Personnel,  Navy  school  officials,  and  the  research  community. 


Kathleen  E.  Moreno 
Department  Director,  . 
Personnel  and  Organizational 
Assessment 
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Summary 


Problem 

This  study  was  conducted  in  response  to  a  request  from  the  Chief  of  Naval  Personnel  (PERS- 
23)  to  validate  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  and  the  joint-service 
English  Diagnostic  Test  (EDT)  as  selection  criteria  for  the  Basic  Journalist  (JO)  Class  “A”  School. 
The  study  was  requested  in  review  of  a  proposal  to  administer  the  EDT  at  the  Military  Entrance 
Processing  Stations  (MEPS)  as  a  selection  screen  rather  than  at  the  JO  school  as  a  remediation  tool. 

The  ASVAB  consists  of  the  following  10  tests:  General  Science  (GS),  Arithmetic  Reasoning 
(AR),  Word  Knowledge  (WK),  Paragraph  Comprehension  (PC),  Numerical  Operations  (NO), 
Coding  Speed  (CS),  Auto  and  Shop  Information  (AS),  Mathematics  Knowledge  (MK), 
Mechanical  Comprehension  (MC),  and  Electronics  Information  (El).  Verbal  (VE)  is  comprised  of 
WK  and  PC.  The  EDT  is  a  test  of  grammar,  comprehension,  word  knowledge,  and  sentence 
structure. 

Objective 

The  objectives  of  this  research  were  to  (1)  validate  the  operational  ASVAB  selector  composite 
against  JO  school  performance  measures,  (2)  identify  and  evaluate  alternative  ASVAB  composites 
that  may  be  more  effective  for  determining  qualification  for  school  assignment,  (3)  determine  a 
minimum  qualifying  score  for  the  recommended  ASVAB  selector  composite  that  would  reduce 
attrition,  and  (4)  assess  the  EDT  as  a  school  selection  standard  and  as  a  remediation  screen. 

Approach 

The  JO  students  were  randomly  assigned  to  a  test  selection  sample  (60%)  and  a  holdout  sample 
(40%).  Two  methods  applying  multiple  regression  were  used  with  the  test  selection  sample  to 
develop  an  experimental  ASVAB  composite  most  predictive  of  final  school  grade  (FSG).  Both 
methods  developed  a  prediction  equation  starting  with  the  ASVAB  test  having  the  highest 
correlation  with  FSG,  followed  by  tests  that  provided  the  largest  increase  in  the  multiple 
correlation.  The  first  four  tests  to  enter  the  equation  were  designated  as  the  experimental  composite 
(the  maximum  used  in  military  ASVAB  composites).  Method  I  did  not  correct  for  restriction  in 
range  of  ASVAB  test  scores  resulting  from  ASVAB  selection,  whereas  Method  II  did. 

The  operational  and  experimental  selector  composites  were  cross-validated  in  the  holdout 
sample.  Validities  were  compared  after  correcting  ASVAB  scores  for  restriction  in  range.  The 
operational  selector  composite  warranted  replacement  if  the  experimental  composite  demonstrated 
a.05  increase  in  validity  or  a  2%  expected  increase  in  the  graduation  rate. 

The  total  JO  sample  was  then  used  to  compare  the  validities  of  the  EDT  and  the  operational 
ASVAB  selector  composite  (or  a  replacement  if  one  was  warranted).  The  criteria  for 
recommending  EDT  as  an  operational  selector  composite  were  the  same  as  the  criteria  for 
replacing  the  operational  ASVAB  selector  composite. 
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Minimum  qualifying  scores  for  the  recommended  ASVAB  selector  composite  and  for  the  EDT 
were  evaluated  from  expectancy  tables,  which  show  the  impact  of  raising  or  lowering  scores  on 
attrition.  Recommendations  for  minimum  qualifying  scores  were  based  upon  (1)  attrition  rate,  (2) 
waiver  rate,  (3)  yearly  school  input  requirement,  and  (4)  the  number  of  graduates  versus  the 
number  of  attrites  disqualified  from  the  JO  school. 

Results  and  Conclusions 

The  operational  ASVAB  selector  composite  (VE  +  AR)  was  adequate  for  the  JO  school. 
Raising  the  minimum  qualifying  score  from  110  to  115  increased  the  graduation  rate  by  3%  for 
remediated  students  (from  84%  to  87%)  while  qualifying  approximately  25%  of  the  recruit 
population  for  school  selection.  The  EDT  had  higher  validity  than  the  ASVAB.  An  EDT  score  of 
70  resulted  in  a  95%  expected  graduation  rate  and  a  25%  expected  qualifying  rate  for  the  recruit 
population.  Because  the  correlation  (corrected)  between  the  EDT  and  VE  +  AR  was  less  than  one 
(.78),  the  EDT  used  as  a  second  stage  selection  screen  would  qualify  less  than  25%  of  the  recruit 
population,  whereas  using  the  EDT  as  an  alternative  selection  standard  would  qualify  more. 

Recommendations 

The  following  recommendations  regarding  the  Basic  JO  Class  “A”  selection  standards  are 
addressed  to  PERS-23: 

1.  The  operational  ASVAB  selector  composite,  VE  +  AR,  should  be  retained  for  school 
selection.  The  minimum  qualifying  score  should  be  raised  from  110  to  115  to  reduce  attrition. 

2.  The  EDT  should  be  implemented  as  a  JO  school  selection  standard  where  logistics  deter¬ 
mine  test  administration  is  most  cost  effective.  The  minimum  qualifying  score  should  be  70.  In 
preparation  for  the  test,  self-paced  materials  covering  EDT  content  areas  should  be  issued  to 
enlistees  interested  in  the  JO  rating. 

3.  The  J-Prep  curriculum  should  remain  intact  for  waivers  (probably  5  score  points  for 
VE  +  AR  and  5  score  points  for  the  EDT)  because  they  may  be  needed  to  fill  school  input  require¬ 
ments. 

4.  Further  analyses  should  be  conducted  for  the  JO  school  when  sufficient  data  become 
available  to  determine  the  impact  of  implementing  the  EDT  as  a  second  stage  selection  standard, 
and  to  determine  if  the  VE  +  AR  and  EDT  minimum  qualifying  scores  are  appropriate. 

Elimination  of  J-Prep  cannot  be  recommended  until  an  assessment  of  the  self-paced 
remediation  materials  has  been  made  and  the  impact  of  implementation  of  the  two  selection 
standards  has  been  determined.  Depending  on  the  expected  impact  of  downsizing,  changes  in  the 
recruit  ability  distribution,  and  forecasts  in  yearly  school  input  requirements,  the  two  selection 
standards  used  in  conjunction  may  eliminate  valuable  recruit  talent  from  the  JO  school.  In  that 
case,  the  ASVAB  and  EDT  could  be  better  utilized  as  either/or  selection  standards  rather  than  first 
and  second  stage  screens  (undetermined  from  this  study  because  all  JO  students  were  required  to 
pass  the  EDT).  The  JO  school  could  develop  an  assessment  program  to  evaluate  (1)  self-paced 
materials  followed  by  the  EDT  test  versus  (2)  the  EDT  test  followed  by  J-Prep  followed  by  EDT 
test. 
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Introduction 


Background  and  Problem 

The  Chief  of  Naval  Personnel  (PERS-23)  tasked  the  Navy  Personnel  Research  and 
Development  Center  to  validate  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB) 
selection  standards  and  the  English  Diagnostic  Test  (EDT)  for  the  joint-service  Basic  Journalist 
(JO)  Class  “A”  school.  Concern  about  attrition  prompted  this  study. 

The  ASVAB,  the  selection  and  classification  instrument  for  all  the  military  services,  consists 
of  the  following  10  tests:  General  Science  (GS),  Arithmetic  Reasoning  (AR),  Word  Knowledge 
(WK),  Paragraph  Comprehension  (PC),  Numerical  Operations  (NO),  Coding  Speed  (CS),  Auto 
and  Shop  Information  (AS),  Mathematics  Knowledge  (MK),  Mechanical  Comprehension  (MC), 
and  Electronics  Information  (El).  These  tests,  described  briefly  in  Table  1,  are  used  by  each  service 
in  various  test  combinations  (composites)  to  select  recruits  into  military  occupations.  The  Navy  has 
11  operational  ASVAB  selector  composites,  listed  in  Table  2.  Periodically  ASVAB  selector 
composites  are  correlated  with  school  performance  measures  (validated)  for  each  Navy  rating  to 
ensure  the  Class  “A”  school  is  using  the  most  effective  ASVAB  selector  composite.  The 
operational  ASVAB  selector  composite  for  the  JO  “A”  school  is  VE  +  AR  (General  Technical 
composite).1  The  minimum  qualifying  score  is  110. 

The  JO  “A”  school  administers  a  school  developed  screening  instrument,  the  English 
Diagnostic  Test  (EDT),  to  assess  proficiency  in  grammar,  comprehension,  word  knowledge,  and 
sentence  structure.  The  EDT  is  administered  prior  to  attending  the  JO  school.  Students  scoring  less 
than  75  on  the  EDT  are  required  to  attend  an  approximately  2- week  remediation  course  termed 
J-Prep.  Upon  completion  of  J-Prep,  students  are  retested  and  must  score  at  least  75  in  order  to 
proceed  to  the  JO  school. 

The  EDT  is  being  evaluated  in  this  study  because  the  JO  school  reports  it  has  high  validity  in 
predicting  attrition,  particularly  in  the  Journalism  component  of  the  curriculum  (first  11  weeks). 
Very  little  attrition  occurs  in  the  follow-on  component.  Broadcasting,  Shipboard  Information  and 
Training  (approximately  5  weeks  but  will  increase  subsequent  to  this  study). 

The  Navy  EDT  minimum  qualifying  score  for  remediation  was  lowered  from  75  to  70 
subsequent  to  this  study’s  data  collection.  There  is  concern  that  70  is  too  low  because  high  attrition 
occurred  when  70  was  used  prior  to  April  1991.  There  is  also  concern  that  the  EDT  should  be 
administered  as  a  Navy  selection  standard  (as  is  done  for  the  Marines,  with  a  required  score  of  70) 
at  the  Military  Entrance  Processing  Stations  (MEPS).  This  would  eliminate  the  costs  of 
transportation,  housing,  remediation,  and  “down  time”  for  students  who  are  sent  to  the  JO  “A” 
school  but  fail  to  qualify. 


’Word  Knowledge  (WK)  and  Paragraph  Comprehension  (PC)  are  combined  into  the  ASVAB  Verbal  (VE)  composite. 
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Table  1 


Content  of  ASVAB  Tests 


Test 

Abbreviation  Description 

General  Science 

GS 

A  25-item  test  of  knowledge  of  the  physical  (13 
items)  and  biological  (12  items)  sciences-- llminutes. 

Arithmetic  Reasoning 

AR 

A  30-item  test  of  ability  to  solve  arithmetic  word 
problems— 36  minutes. 

Word  Knowledgea 

WK 

A  35-item  test  of  knowledge  of  vocabulary,  using 
words  embedded  in  sentences  (11  items)  and 
synonyms  (24  items)- 11  minutes. 

Paragraph  Comprehension 

PC 

A  15-item  test  of  reading  comprehension-13 
minutes. 

Numerical  Operations 

NO 

A  50-item  speed  test  of  ability  to  add,  subtract, 
multiply,  and  divide  one-  and  two-digit  numbers-3 
minutes. 

Coding  Speed 

CS 

An  84-item  speeded  test  of  ability  to  recognize 
numbers  associated  with  words  from  a  table— 7 
minutes. 

Auto  &  Shop  Information 

AS 

A  25-item  test  of  knowledge  of  automobiles,  shop 
practices,  and  use  of  tools- llminutes. 

Mathematics  Knowledge 

MK 

A  25-item  test  of  knowledge  of  algebra,  geometry, 
fractions,  decimals,  and  exponents— 24  minutes. 

Mechanical  Comprehension 

MC 

A  25-item  test  of  knowledge  of  mechanical  and 
physical  principles- 19  minutes. 

Electronics  Information 

El 

A  20-item  test  of  knowledge  of  electronics,  radio  and 
electrical  principles  and  information-9  minutes. 

“Verbal  score:  VE  =  WK  +  PC  (raw  scores). 

Table  2 

Navy  Operational  ASVAB  Selector  Composites 

Composite 

Components 

General  Technical 

VE  +  AR 

Mechanical 

VE  +  MC  +  AS 

Electronics 

AR  +  MK  +  EI  +  GS 

Clerical 

VE  +  NO  +  CS 

Basic  Electricity  &  Electronics 

AR  +  2MK  +  GS 

Engineering 

MK  +  AS 

Cryptologic  Technician 

VE+AR  +  NO  +  CS 

Hospitalman 

VE  +  MK  +  GS 

Machinery  Repairman 

AR  +  MC  +  AS 

Submarine 

VE  +  AR +  MC 

Business/Clericala 

VE  +  MK  +  CS 

Note.  See  Table  1  for  full  test  names. 

“Student  Testing  Program  composite  implemented  July  1987. 
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Objectives 


The  objectives  of  this  study  were  to  (1)  validate  the  operational  ASVAB  selector  composite 
against  JO  “A”  school  performance  measures,  (2)  identify  and  evaluate  alternative  ASVAB 
composites  that  may  be  more  effective  for  determining  qualification  for  school  assignment,  (3) 
determine  a  minimum  qualifying  score  for  the  recommended  ASVAB  selector  composite  that 
would  reduce  attrition,  and  (4)  assess  the  EDT  as  a  school  selection  standard  and  as  a  remediation 
screen. 


Approach 


Predictors 

The  predictors  were  the  10  tests  of  ASVAB  Forms  8  through  14  and  the  EDT.  The  ASVAB  is 
described  briefly  in  Table  1.  Raw  test  scores  were  standardized  to  a  mean  of  50  and  a  standard 
deviation  of  10  using  norms  obtained  from  the  American  Youth  Population  (Department  of 
Defense,  1982).  The  EDT  scores,  provided  by  the  JO  school,  were  on  a  0  to  100  scale. 

Criteria 

The  criteria  were  final  school  grade  (FSG)  and  scores  on  the  Journalism  component  of  the  JO 
school  curriculum.  Scores  for  both  criteria,  provided  by  the  JO  school,  were  on  a  0  to  100  scale. 
Attrites  were  retained  for  analyses  to  provide  a  representative  sample  of  all  JO  students.  A 
mathematical  procedure  developed  by  Abrahams  and  Alf  (1992)  and  detailed  in  Appendix  A 
estimated  FSGs  for  attrites. 

Samples 

The  data,  collected  from  September  1990  to  May  1993,  were  obtained  from  the  JO  “A”  school. 
The  final  sample  size  was  243;  26  academic  drops  and  217  graduates  (2  nonacademic  drops  were 
deleted  from  the  sample). 

Data  Analyses 

The  JO  students  were  randomly  assigned  to  a  test  selection  sample  (60%)  and  a  holdout  sample 
(40%).  Prior  to  this  assignment,  students  were  sorted  by  Student  Action  Code  (SAC)  to  ensure  that 
each  sample  had  equal  percentages  of  graduates  and  attrites. 

Two  methods  applying  multiple  regression  were  used  with  the  test  selection  sample  to  develop 
an  experimental  ASVAB  composite  most  predictive  of  FSG.  Both  methods  developed  a  prediction 
equation  starting  with  the  ASVAB  test  having  the  highest  correlation  with  FSG,  followed  by  tests 
that  provided  the  largest  increase  in  the  multiple  correlation.  The  first  four  tests  to  enter  the 
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equation  were  designated  as  the  experimental  composite.  Method  I  did  not  correct  for  restriction 
in  range  of  ASVAB  test  scores  resulting  from  ASVAB  selection,  whereas  Method  II  did. 

The  operational  and  experimental  selector  composites  were  cross-validated  (against  FSG)  in 
the  holdout  sample  using  integer  weights  of  one  for  each  test.2 3  Composite  validities  were  compared 
after  correcting  for  restriction  in  range  (to  obtain  the  validity  for  a  typical  recruit  applicant 
population,  rather  than  a  selected  sample).  A  multivariate  correction  procedure  (Lawley,  1943)  was 
used  for  Method  II  and  is  explained  in  Appendix  B.  Results  from  the  multiple  regression  analyses 
for  Methods  I  and  II  are  in  Appendix  C.  Candidate  ASVAB  replacement  composites  were 
evaluated  when  the  experimental  composite  demonstrated  (1)  a  .05  increase  in  the  validity  or  (2)  a 
2%  expected  increase  in  the  graduation  rate.4 

The  total  JO  sample  was  then  used  to  evaluate  the  EDT  relative  to  the  ASVAB  in  predicting 
both  FSG  and  the  Journalism  component  of  the  curriculum.  The  criteria  for  including  the  EDT  as 
a  formal  selection  standard  were  those  used  to  evaluate  the  ASVAB  (a  .05  increase  in  the  validity 
or  a  2%  expected  increase  in  the  graduation  rate). 

Minimum  qualifying  scores  for  the  recommended  ASVAB  selector  composite  and  for  the  EDT 
were  evaluated  from  expectancy  tables.  Expectancy  tables  show  expected  changes  in  the 
graduation  rate  when  the  minimum  qualifying  score  is  raised  or  lowered.  The  school  data  are  used 
to  develop  the  expectancy  table  for  the  operational  composite  (if  the  recommendation  is  to  retain 
it  for  school  selection).  The  theory-based  Taylor-Russell  tables  are  used  if  there  is  a  recommended 
replacement,  and  for  the  EDT.  Theory-based  expectancy  tables  are  used  to  evaluate  a  replacement 
ASVAB  selector  composite  (and  an  added  selection  instrument  in  this  study)  rather  than  empirical 
expectancy  tables  because  improvements  in  the  graduation  rate  cannot  be  adequately  evaluated  for 
students  who  have  already  been  selected  by  the  correlated  ASVAB  operational  selector  composite. 

The  impact  of  raising  the  EDT  score  from  70  to  75  (mid-FY91)  on  attrition  was  assessed  from 
summary  data  obtained  from  the  Navy  Integrated  Training  Resources  and  Administration  System 
(NITRAS)  for  a  4-year  period. 

The  following  factors  were  considered  in  evaluating  minimum  qualifying  scores:  (1)  attrition 
rate,  (2)  waiver  rate,  (3)  yearly  school  input  requirement,  and  (4)  the  number  of  graduates  versus 
the  number  of  attrites  disqualified  from  the  JO  school. 


2Fout  tests  are  designated  as  the  experimental  composite,  rather  than  the  tests  that  provide  significant  increases  in  predic¬ 
tion,  because  four  is  the  maximum  number  of  tests  used  in  the  eleven  NAVY  operational  ASVAB  selector  composites. 
Unless  the  experimental  composite  has  been  derived  repeatedly  for  schools  of  an  occupational  group,  the  Navy  ASVAB 
composite  most  similar  to  the  experimental  composite  is  recommended  for  school  selection  (if  the  validities  for  both 
composites  warrant  the  change). 

integer  weights  generalize  to  cross-validation  samples  more  successfully  than  exact  weights  derived  from  regression 
analysis  (Dawes  &  Corrigan,  1974).  However,  regression  weights  determine  optimal  assignment  in  models  of  classifi¬ 
cation  efficiency  (Brogden,  1955). 

4The  Taylor-Russell  tables  (Taylor  &  Russell,  1939)  were  used  to  translate  gain  in  validity  into  expected  gain  in  the  grad¬ 
uation  rate. 
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Results  and  Conclusions 


AS  VAB  and  EDT  Validity 

The  experimental  composite,  VE  +  AR  +  MK  +  NO,  was  identified  in  the  test  selection  sample 
by  both  Methods  I  and  II.  The  cross-validity  in  the  holdout  sample  (all  reported  validities  are 
corrected  for  restriction  in  range)  was  higher  for  the  operational  composite  (.73)  than  for  the 
experimental  composite  (.69).  Therefore,  replacing  the  operational  ASVAB  selector  composite  is 
not  warranted. 

The  comparison  of  validities  for  VE  +  AR  and  the  EDT  was  made  using  the  total  sample 
( N  -  243).  Table  3  lists  the  validities  for  VE  +  AR  and  the  EDT  using  both  FSG  and  the  Journalism 
component  of  the  JO  school  as  criteria. 


Table  3 

VE  +  AR  and  English  Diagnostic  Test  (EDT)  Validities 
with  FSG  and  Journalism  Component  as  Criteria 


Predictors 

Criteria 

VE  +  AR  EDT 

FSG 

.71  .75 

Journalism  Component 

.67  .70 

Note.  All  validities  have  been  corrected  for  restriction  in  range  using 
multivariate  procedures. 


For  predicting  FSG,  the  validity  of  .75  for  the  EDT  was  .04  higher  than  the  validity  of  .71  for 
the  operational  selector  composite,  VE  +  AR.  This  gain  in  validity  translates  into  a  near  2% 
increase  in  the  expected  graduation  rate  and  warrants  consideration  of  including  the  EDT  as  a  JO 
school  selection  standard. 

For  predicting  the  Journalism  component  of  the  curriculum,  the  validity  of  .70  for  the  EDT  was 
.03  higher  than  the  validity  of  .67  for  the  operational  selector  composite,  VE  +  AR.  This  gain  in 
validity  is  consistent  with  the  gain  obtained  from  the  VE  +  AR/EDT  comparison  using  FSG  as  the 
criterion  and  provides  additional  support  for  including  the  EDT  as  a  JO  school  selection  standard. 

Minimum  Qualifying  Scores:  VE  +  AR 

Table  4  is  an  expectancy  table  used  to  evaluate  VE  +  AR  minimum  qualifying  scores.  The 
analysis  was  performed  for  129  students  who  scored  less  than  75  on  the  EDT  and  attended  J-Prep. 
This  group  was  analyzed  instead  of  the  entire  JO  sample  ( N  =  243)  for  two  reasons.  First,  the 
impact  of  raising  the  VE  +  AR  minimum  qualifying  score  on  attrition  cannot  be  accurately  assessed 
for  the  entire  JO  sample  because  some  benefited  from  J-Prep  whereas  others  did  not.  Second, 
attrition  for  students  scoring  75  or  greater  on  the  EDT  was  insignificant. 


5 


Table  4 


Expectancy  Table  for  the  Operational  Selector  Composite  (VE  +  AR) 


Composite 

Score 

School  Sample 

At  or  Above 
Composite  Score  in 
Recruit  Population 

Expectancies  per 
1,000  Recruits 

Grad 

N 

Drop 

N 

Total 

N 

Grad 

% 

Drop 

% 

% 

Total 

% 

Grad 

% 

Drop 

% 

>128 

2 

0 

2 

100 

6 

>  . 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

. 

• 

• 

• 

>120 

34 

2 

36 

94 

6 

14 

140 

132 

8 

>119 

38 

4 

42 

91 

9 

16 

160 

146 

14 

£118 

46 

5 

51 

90 

10 

19 

190 

171 

19 

>117 

51 

5 

56 

91 

9 

21 

210 

191 

19 

>116 

60 

8 

68 

88 

12 

24 

240 

211 

29 

>115a 

74 

11 

85 

87 

13 

27 

270 

235 

35 

>114 

80 

14  . 

94 

85 

15 

29 

290 

247 

43 

>113 

87 

15 

102 

85 

15 

32 

320 

272 

48 

£112 

91 

15 

106 

86 

14 

35 

350 

301 

49 

>111 

100 

17 

117 

86 

14 

38 

380 

327 

53 

£ll0b 

103 

19 

122 

84 

16 

41 

410 

344 

66 

£109 

104 

19 

123 

85 

15 

44 

440 

374 

66 

£108 

107 

20 

127 

84 

16 

47 

470 

395 

75 

£107 

108 

20 

128 

84 

16 

50 

500 

420 

80 

£106 

108 

20 

128 

84 

16 

53 

530 

445 

85 

£105 

108 

20 

128 

84 

16 

56 

560 

470 

90 

£  . 

• 

• 

• 

• 

• 

• 

• 

• 

• 

£  • 

. 

. 

. 

• 

• 

. 

. 

• 

• 

£102 

108 

21 

129 

84 

16 

65 

650 

546 

104 

Note.  N  =  129  remediated  students.  Of  the  7  students  who  scored  below  the  110  minimum  qualifying  score  (129  - 
122  =  7  waivers),  2  (29%)  were  attrites.  Attrites  are  designated  as  drops. 

Alternative  minimum  qualifying  score. 
bCurrent  minimum  qualifying  score. 

Table  4  lists  a  range  of  operational  selector  composite  minimum  qualifying  scores  that  include 
the  110  score  currently  used.  A  breakdown  for  each  score  includes  actual  graduation  and  attrition 
rates  (attrites  are  designated  as  drops)  for  the  school  sample  (N=  129)  and  expected  rates  (per  1,000) 
for  the  recruit  population  ( N  =  58,825).  Expected  rates  are  based  on  school  rates. 

Attrition  was  16%  for  those  scoring  at  or  above  the  current  110  minimum  qualifying  score. 
Raising  the  minimum  qualifying  score  to  115  improved  the  graduation  rate  by  3%  (from  84%  to 
87%),  but  at  the  cost  of  eliminating  29  graduates  from  school  selection.  However,  the  JO  “A” 
school  could  benefit  from  this  raise  considering  (1)  the  waiver  rate  is  low  (3%,  or  7/243),  (2)  the 
yearly  student  input  requirement  relative  to  other  Class  “A”  schools  is  low  (approximately  150  for 
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FY93),  and  (3)  the  percentage  of  the  recruit  population  that  would  qualify  should  be  sufficient 
(27%  for  FY92). 

Minimum  Qualifying  Scores:  EDT  as  a  Selection  Standard 

Table  5  is  the  theory-based  expectancy  table  used  to  evaluate  EDT  minimum  qualifying  scores 
and  expected  graduation  rates. 


Table  5 

Expectancy  Table  for  the  English  Diagnostic  Test  (EDT) 


EDT  Score 

At  or  Above  EDT  in 
Population 

Expectancies  in  Population 

% 

% 

% 

>79 

10 

99 

1 

>75 

15 

98 

2 

*73 

20 

96 

4 

>70 

25 

95 

5 

>69 

30 

93 

7 

£67 

35 

92 

8 

>65 

40 

90 

10 

>63 

45 

88 

12 

£61 

50 

86 

14 

Note.  This  table  was  derived  from  the  theoretically  based  Taylor-Russell  tables  (Taylor  &  Russell, 

1939)  and  values  obtained  from  multivariate  correction  for  range  restriction  procedures  using  the 
Navy  applicant  population  (see  Appendix  B).  The  values  are  only  expected  values  (actual 
population  EDT  values  are  not  available  for  a  population).  The  applicant  population  was  expected 
to  produce  slightly  different  results  than  the  recruit  population. 

Table  5  shows  that  implementing  an  EDT  minimum  qualifying  score  of  75  (presumed  before 
remediation)  results  in  an  expected  98%  graduation  rate.  However,  only  15%  of  the  recruit 
population  qualify  for  school  selection.  The  JO  school,  having  relatively  low  student  input 
requirements,  can  afford  to  be  selective,  however  several  factors  must  be  considered:  (1)  Using 
EDT  =  75  as  a  selection  standard  rather  than  a  remediation  diagnostic  would  have  eliminated  over 
half  of  the  graduates  in  this  study  from  school  selection,  (2)  JO  requirements  that  are  too  stringent 
will  create  shortages  of  JOs  in  the  field,  and  (3)  despite  military  downsizing,  the  demand  for  JOs 
is  still  stable  (Navy  input  requirements  for  FY94  are  greater  than  150). 

An  EDT  minimum  qualifying  score  of  70  (now  implemented  in  the  JO  school  as  a  remediation 
standard)  results  in  an  expected  95%  graduation  rate  and  qualifies  25%  of  the  recruit  population 
for  school  selection  (close  to  the  27%  qualified  with  VE  +  AR  =  115).  Because  the  correlation 
(corrected)  between  the  EDT  and  VE  +  AR  was  less  than  one  (.78),  the  EDT  used  as  a  second  stage 
selection  screen  would  qualify  less  than  25%  of  the  recruit  population,  whereas  using  the  EDT  as 
an  alternative  selection  standard  would  qualify  more. 
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Minimum  Qualifying  Scores:  EDT  as  a  Remediation  Diagnostic 

Table  6  lists  attrition  rates,  setback  rates,  and  planned  input  requirements  for  FY89  through 
FY92  (obtained  from  NITRAS). 


Table  6 


JO  Attrition  Rates,  Setbacks,  and  Input  Requirements  for  FY89-92 


Fiscal 

Year 

Attrition  Rate  (%) 

Setbacks 

Planned  Input 
Requirement 

Academic 

Nonacademic 

Total 

1989 

13 

6 

19 

34 

96 

1990 

.  12 

9 

21 

50 

118 

1991a 

12 

1 

13 

17 

129 

1992 

9 

1 

10 

10 

161 

Note.  Data  obtained  from  Navy  Integrated  Training  Resources  and  Administration  System  (NITRAS). 

aEDT  was  raised  from  70  to  75  April  1991. 

Table  6  shows  academic  attrition  was  reduced  by  3%  in  FY92,  the  last  FY  following  the  raise 
in  the  EDT  minimum  qualifying  score  from  70  to  75.  The  recent  reinstatement  of  EDT  =  70  could 
result  in  a  3%  increase  in  attrition  if  all  other  things  remain  equal  (i.e.,  ASVAB  selection  standard, 
EDT  use,  recruit  ability  distribution,  motivation,  and  yearly  input  requirements). 

Recommendations 

The  following  recommendations  regarding  the  Basic  JO  Class  “A”  selection  standards  are 
addressed  to  PERS  23: 

1.  The  operational  ASVAB  selector  composite,  VE  +  AR,  should  be  retained  for  school 
selection.  The  minimum  qualifying  score  should  be  raised  from  110  to  115  to  reduce  attrition. 

2.  The  EDT  should  be  implemented  as  a  JO  school  selection  standard  where  logistics  deter¬ 
mine  test  administration  is  most  cost  effective.  The  minimum  qualifying  score  should  be  70.  Self- 
paced  materials  covering  EDT  content  areas  should  be  issued  to  enlistees  interested  in  the  JO  rat¬ 
ing  in  preparation  for  the  test. 

3.  The  J-Prep  curriculum  should  remain  intact  for  waivers  (probably  5  score  points  for 
VE  +  AR  and  5  score  points  for  the  EDT)  because  they  may  be  needed  to  fill  school  input  require¬ 
ments. 

4.  Further  analyses  should  be  conducted  for  the  JO  school  when  sufficient  data  become 
available  to  determine  the  impact  of  implementing  the  EDT  as  a  second  stage  selection  standard, 
and  to  determine  if  the  VE  +  AR  and  EDT  minimum  qualifying  scores  are  appropriate. 

Elimination  of  J-Prep  cannot  be  recommended  until  an  assessment  of  the  self-paced 
remediation  materials  has  been  made  and  the  impact  of  implementation  of  the  two  selection 
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standards  has  been  determined.  Depending  on  the  expected  impact  of  downsizing,  changes  in  the 
recruit  ability  distribution,  and  forecasts  in  yearly  school  input  requirements,  the  two  selection 
standards  used  in  conjunction  may  eliminate  valuable  recruit  talent  from  the  JO  school.  In  that 
case,  the  ASVAB  and  EDT  could  be  better  utilized  as  either/or  selection  standards  rather  than  first 
and  second  stage  screens  (undetermined  from  this  study  because  all  JO  students  were  required  to 
pass  the  EDT).  The  JO  school  could  develop  an  assessment  program  to  evaluate  (1)  self-paced 
materials  followed  by  the  EDT  test  versus  (2)  the  EDT  test  followed  by  J-Prep  followed  by  EDT 
test. 
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Appendix  A 
Scoring  Failures 


A-0 


Scoring  of  Failures 


The  scoring  of  failures  procedure  is  based  on  the  assumption  that,  for  an  applicant  population, 
criterion  scores  for  successes  (graduates)  and  failures  (attrites)  when  combined  are  normally 
distributed.  A  mathematical  procedure  (Abrahams  &  Alf,  1992)  is  used  to  assign  failure  scores  at 
the  appropriate  lower  point  of  the  criterion  distribution.  The  following  values  and  formulas  are 
used  for  the  procedure. 

p  =  the  proportion  of  graduates, 

q  =  the  proportion  of  attrites. 

Xg  =  the  mean  final  school  grade  for  the  graduates. 

SDg  =  the  standard  deviation  of  FSGs  for  graduates, 

z  =  the  z-score  (standard  score)  above  which  the  proportion  p  falls, 
y  =  the  height  of  the  normal  curve  at  z. 


Step  1 

The  mean  for  attrites,  Xa,  can  be  determined  as  follows: 
Xa  =  Xg-A(SDg),  where 

A_  y/(pq) 

V 1+  (zy/p)  -  (y/p)2 


Step  2 

Assign  the  estimated  mean  criterion  score  determined  in  Step  1  to  each  attrite. 

Step  3 

Compute  the  correlation  between  each  predictor  and  the  criterion  for  the  combined  distribution 
of  graduates  and  attrites. 

Step  4 

Correct  the  correlations  from  Step  3  for  coarse  grouping  (assigning  a  mean  criterion  score  to 
every  attrite  reduces  variance  and,  therefore,  the  correlation  coefficient).  The  formula  used  for  this 
correction  is: 

rc=  rxy  /SD/,  where 
SD'  =  Vl  -  q  +  zy  +  y2 /q 


A-l 


Appendix  B 

Correction  Procedure  Used  in  Method  II 


B-0 


Correction  Procedure  Used  in  Method  II 


Regression  analysis  used  to  derive  the  ASVAB  composite  most  predictive  of  final  school  grade 
(FSG)  is  biased  against  the  tests  of  the  school  selector  composite  or  tests  highly  correlated  to  them. 
This  bias  is  known  as  restriction  in  range.  Correcting  test  scores  for  restriction  in  range  is 
accomplished  in  Method  II  by  using  a  Navy  applicant  population  ASVAB/FSG  matrix  where 
correlations  between  ASVAB  tests  and  FSG  are  estimated  using  the  general  multivariate  correction 
formulas  (Lawley,  1943).  The  ASVAB  tests  are  treated  as  explicit  selector  variables  (known  values 
for  the  population  and  restricted  sample),  whereas  FSG  is  treated  as  the  incidental  selector  variable 
(known  values  for  the  restricted  sample  only). 

Table  B-l  gives  two  matrices  (including  means  and  standard  deviations)  required  for  the 
multivariate  correction  procedure.  The  first  is  the  ASVAB/FSG  matrix  for  the  JO  test  selection 
sample  (see  Table  1  for  the  full  test  names).  The  second  is  an  ASVAB  matrix  for  the  FY92  Navy 
applicant  population  ( N  =  115,986).  At  the  bottom  of  the  page  are  the  estimated  population 
correlations  between  ASVAB  and  FSG. 

The  correction  procedure  to  determine  the  EDT  validity  for  the  total  sample  used  the  ASVAB 
as  explicit  selection  variables,  and  the  EDT,  Journalism  component,  and  FSG  as  incidental 
selection  variables  (unknown  for  the  population).  In  addition  to  corrected  validities,  the  correction 
procedure  also  calculates  expected  population  means  and  standard  deviations  for  the  incidental 
selection  variables.  The  EDT  means  and  standard  deviations  were  used  to  construct  Table  5  (EDT 
expectancy  analysis). 
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Table  B-l 


Required  Multivariate  Matrices  and  Output 


GS 

AR 

CS 

AS 

MK 

El 

VE 

FSG 

Mean 

SD 

ET-NF  BE&E  Test  Selection  Sample  Matrix  With  Means  and  Standard  Deviations 

GS 

1.000 

.361 

-.010 

-.061 

.351 

.374 

.495 

.440 

.216 

57.53 

4.28 

AR 

1.000 

.182 

.131 

.282 

.604 

.370 

.340 

.149 

.356 

59.84 

2.58 

NO 

1.000 

.504 

-.131 

.218 

-.020 

-.108 

l 

o 

►— * 

.223 

56.66 

5.36 

CS 

1.000 

-.120 

.147 

-.053 

-.114 

.040 

.088 

55.95 

7.04 

AS 

1.000 

.068 

.467 

.589 

.161 

.125 

51.97 

6.29 

MK 

1.000 

.315 

.168 

.162 

.351 

59.52 

2.74 

MC 

1.000 

.495 

.202 

.167 

56.92 

5.53 

El 

1.000 

.257 

.150 

52.83 

5.86 

VE 

1.000 

.198 

58.49 

3.00 

FSG 

1.000 

81.21 

4.95 

Population  (Applicant  FY 92)  With  Means  and  Standard  Deviations 


GS 

1.000 

.567 

.189 

51.66 

AR 

1.000 

.405 

.340 

.376 

.695 

.580 

.442 

.571 

51.97 

8.20 

NO 

1.000 

.600 

.020 

.430 

.158 

.093 

.273 

53.41 

7.42 

CS 

1.000 

.029 

.355 

.165 

.097 

.294 

52.64 

7.46 

AS 

1.000 

.201 

.599 

.634 

.386 

50.78 

8.72 

MK 

1.000 

.479 

.357 

.490 

53.56 

8.24 

MC 

1.000 

.603 

.507 

52.70 

8.75 

El 

1.000 

.476 

50.42 

8.40 

VE 

1.000 

52.67 

6.44 

Correlations  (Validities)  for  Population  From  Correction  Program  and  Above  Matrices 

FSG 

.500 

.630 

.434 

.311 

.295 

.585 

.428 

.362 

.585 
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Appendix  C 

Multiple  Regression  for  Methods  I  and  II 
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Multiple  Regression  for  Methods  I  and  II 


STEP 

TEST 

MULTR 

RSQ 

F 

FSIG 

RSQCH 

FCH 

SIGCH 

REG-DF 

RES-DF 

JO  Test  Selection  Sample  Method  I  (AR  +  MK  +  NO  +  VE) 

i 

AR 

.3519 

.1238 

20.35 

.1238 

.000 

1 

144 

2 

MK 

.1522 

12.84 

4.79 

2 

143 

3 

NO 

.4137 

.1711 

9.77 

3.24 

3 

142 

4 

VE 

.4377 

.1916 

8.35 

3.57 

4 

141 

Recruit  Applicant  Population  (FY92)  Method  II  (AR  +  VE  +  NO  +  MK) 

i 

AR 

.6300 

.3969 

.3969 

1 

2 

VE 

.6871 

.4721 

.0752 

2 

115,983 

3 

NO 

.7104 

.5047 

.0326 

3 

115,982 

4 

MK 

.7217 

.5209 

.0161 

4 

115,981 

Note.  See  Table  1  for  full  test  names. 


The  multiple  regression  results  (SPSSX,  1983)  for  Method  I  show  that  MK  is  entered  into  the 
composite  equation  at  Step  2,  at  which  point  the  multiple  correlation  for  the  composite  AR  +  MK 
is  .3902.  The  squared  multiple  correlation  (the  proportion  of  final  school  grade  variance  accounted 
for  by  the  composite)  is  .1522.  The  F  statistic  to  determine  the  significance  of  the  predictive 
relationship  between  the  composite  AR  +  MK  and  final  school  grade  is  12.84.  The  probability  that 
this  predictive  relationship  is  due  to  chance  is  less  than  .001.  The  change  in  the  squared  multiple 
correlation  upon  entering  the  MK  test  into  the  equation  is  .0284.  The  F  statistic  for  change  (to 
determine  the  significance  of  the  increase  in  the  predictive  relationship  by  adding  the  MK  test  into 
the  equation)  is  4.79.  The  probability  that  the  significance  of  this  addition  is  due  to  chance  is  less 
than  or  equal  to  .03.  The  degrees  of  freedom  (number  of  observations  minus  the  number  of 
estimated  parameters)  are  2  for  regression  and  143  for  residual. 

Method  II  is  based  on  corrected  correlations.  Since  there  are  no  appropriate  significance  tests 
for  corrected  correlations,  the  F  tests  for  this  method  do  not  apply. 
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